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Abstract 


U.S.  Army  Cadet  Command  (USACC)  has  developed  a  new  commissioning  model  - 
Shaping  the  Cohort  (STC)  -  to  meet  the  future  needs  of  the  Army  for  commissioned  officers.  It 
is  designed  to  shape  each  cohort  to  meet  the  Army’s  specific  needs  in  terms  of  component, 
academic  disciplines,  race/ethnic  makeup  goals,  gender,  and  targeted  missions.  STC  does  this  by 
determining  and  examining  the  “prime  market”  at  a  university  and  basing  the  detachment’s 
mission  on  penetration  of  that  market  as  opposed  to  one  based  on  past  performance.  It  is 
believed  that  the  STC  model  improves  the  method  of  determining  missions. 

To  determine  market  potential,  USACC  conducted  two  surveys  that  included  62  colleges 
and  universities  and  over  7600  students.  The  goals  of  the  survey  were  to  determine  knowledge 
and  perception  of  Army  ROTC  among  students,  segmentation  of  local  markets,  how  the  school 
markets  differed,  and  the  characteristics  that  could  lead  to  participation  in  Army  ROTC.  The 
data  gathered  is  used  to  determine  how  many  students  at  each  school  fit  the  criteria  for  the  prime 
market. 

The  Operations  Research  Center  of  Excellence  (ORCEN)  at  West  Point  has  provided  an 
independent  assessment  of  the  model’s  adequacy  and  to  determine  if  and  how  it  can  be 
improved.  We  begin  this  report  with  a  needs  analysis  to  determine  the  real  problem  the  model 
attempts  to  address.  We  then  present  our  analysis  of  the  STC  model  and  the  process  for 
determining  missions.  In  the  next  section  we  identify  recommendations  to  improve  the  model 
and  conclude  with  suggested  additional  tools  for  conducting  this  analysis. 
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Chapter  1:  Introduction 

1.1.  A  Brief  History  of  the  Army  Reserve  Officer  Training  Corps 
(ROTC). 

Army  ROTC  is  the  single  largest  commissioning  source  for  second  lieutenants  in  the 
active  Army,  US  Army  Reserve,  and  the  Army  National  Guard.  Between  60  and  70  percent  of 
these  second  lieutenants  are  graduates  of  the  ROTC  program.  ROTC  officially  began  on  3  June 
1916  when  President  Woodrow  Wilson  signed  the  National  Defense  Act  of  1916.  The  first 
group  of  133  cadets  received  their  commissions  during  the  1919-1920  academic  year.  Since 
then,  over  half  a  million  cadets  have  been  commissioned  as  second  lieutenants  at  one  of  the  over 
270  college  and  university  ROTC  programs.  In  1986,  the  US  Army  Cadet  Command  (USACC) 
was  established  as  a  major  subordinate  command  of  the  US  Army  Training  &  Doctrine 
Command  (TRADOC).  In  1998,  cuts  mandated  by  TRADOC  resulted  in  a  major  reorganization 
of  USACC  and  its  three  region  headquarters.  This  began  a  trend  in  USACC  toward  trying  to 
find  ways  to  do  “more  with  less”,  of  which  the  Shaping  the  Cohort  (STC)  model  is  a  product 
(“Brief  History  of  Army  ROTC”). 

1.2.  Background. 

The  Reserve  Officer  Training  Corps,  or  ROTC,  at  college  campuses  throughout  the 
country  has  undergone  significant  changes  in  the  past  few  years  following  the  publication  of  the 
guidance  presented  in  “College  ROTC:  The  Way  Ahead”  (April,  2001,  USACC).  This  policy 
document  moved  college-level  ROTC  programs  away  from  an  attrition-based  model  to  a 
developmental  leadership  model.  Though  this  is  a  significant  departure  for  the  management  of 
the  cadets  in  the  ROTC  programs,  it  did  not  immediately  impact  the  process  of  recruiting  the 
cadets  into  the  program  initially. 

The  US  Army  Cadet  Command  (USACC)  affects  the  recruiting  and  retention  of  cadets  in 
the  national  ROTC  program  most  significantly  through  the  allocation  of  recruiting  goals  and  the 
allocation  of  resources.  Until  recently,  the  USACC  determined  these  commissioning  goals,  or 
missions  almost  exclusively  on  a  school’s  historical  production  rates.  USACC  determined 
resource  allocation  similarly  -  in  other  words,  resources  tend  to  be  allocated  to  places  where  they 
were  used  in  the  past,  without  considering  any  other  factors.  They  soon  realized  that  the  shift  in 
program  styles  required  a  change  in  missioning  philosophy. 
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1.3.  The  Purpose  of  STC. 

Cadet  Command  recognized  that  determining  a  school  commissioning  mission  based 
solely  on  its  historical  production  was  keeping  schools  from  penetrating  into  larger  potential 
markets  that  might  exist  on  campus.  The  Shaping  the  Cohort  (STC)  model  paved  the  way  for  a 
move  toward  a  market  analysis  approach,  where  market  size  was  measured  by  determining  the 
number  of  eligible  students  that  possessed  the  desired  characteristics  and  a  likely  interest  in 
ROTC  participation.  Also,  STC  builds  each  ROTC  cohort  to  meet  certain  targeted  goals  with 
regard  to  gender,  race/ethnicity,  and  academic  discipline  mix  (ADM).  Two  internal  Cadet 
Command  studies  made  this  possible:  the  On-Campus  Market  Potential  Study  (OCMPS)  and  the 
Officer  Accessions  Strategy  (OAS). 

1.3.1.  The  On-Campus  Market  Potential  Study  (OCMPS). 

The  OCMPS  was  conducted  in  April  of  2001  and  2002,  and  surveyed  over  7600  students 
at  62  universities.  The  purpose  of  the  study  was  to  identify  students  who  had  the  characteristics 
desired  for  future  officers  (identified  as  Scholar- Athlete-Leader,  or  SAL)  as  well  as 
characteristics  that  made  them  more  likely  to  participate  in  ROTC  if  they  were  exposed  to  the 
program  (primarily  identified  as  “First-Stringers”1).  Examination  of  the  data  in  this  study 
showed  that  the  two  strongest  predictors  of  the  values  for  SALs  and  First  Stringers  were  the  size 
of  the  school  and  its  academic  reputation.  The  aggregate  data  is  represented  by  fourteen  clusters 
based  on  these  predictors.  Work  on  this  study  is  continuing,  and  is  intended  to  eventually 
encompass  all  272  schools. 

1.3.2.  The  Officer  Accessions  Strategy  (OAS). 

The  other  main  component  of  STC  is  to  directly  tailor  the  shape  of  each  cohort  with 
regard  to  race/ethnicity,  gender,  and  ADM.  The  OAS  determines  the  distribution  of  graduates  at 
colleges  in  terms  of  these  factors.  This  information  is  then  used  to  help  determine  which 
portions  of  the  cohort  an  individual  program  should  be  responsible  for  producing,  which  is 
passed  on  to  the  ROTC  detachment  as  targeted  missions  (in  addition  to  the  overall  production 
mission). 


1  These  numbers  are  expressed  as  percentages  of  the  school  population;  the  SAL  percentage  is  dependent  on  the 
First  Stringers  (i.e  percentage  of  First  Stringers  who  are  also  SALs). 
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1.4.  Approach. 

LTC  William  Warner  (Division  Chief,  Operations  Analysis  Division,  RROD,  USACC) 
briefed  us  on  the  background  and  concept  of  STC  (Warner,  2004),  and  provided  a  copy  of  the 
relevant  computer  files.  We  then  began  our  examination  of  the  model  and  its  implementation. 

An  interview  was  conducted  with  COL  Michael  Hoff  (Director,  Recruiting  &  Retention 
Operations  Directorate,  USACC)  and  LTC  Warner  to  get  their  perspective  on  the  role  of  the 
current  model  and  the  direction  they  thought  it  should  take  to  meet  the  needs  of  USACC.  During 
this  interview,  they  stated  that  MG  Alan  W.  Thrasher  (CG,  USACC)  wants  his  subordinate 
commanders  to  have  the  freedom  to  exercise  their  professional  military  judgment  with  regard  to 
the  final  commissioning  mission  assignments  (Hoff,  2004).  The  implication  of  this  is  that  the 
model  outputs  are  intended  as  a  starting  point,  which  can  be  negotiated  within  the  chain  of 
command  to  a  different  value  if  sufficient  justification  is  given.  Additionally,  they  expressed  a 
desire  to  improve  the  quality  of  their  model  inputs,  and  to  try  to  find  a  measure  of  propensity  (a 
desire  and  commitment  to  join  ROTC)  (Hoff,  2004).  We  then  conducted  a  detailed  analysis  of 
the  model  and  its  implementation,  along  with  an  analysis  of  functions  and  data  stored  in  the 
computer  files.  We  conducted  a  follow-on  interview  with  LTC  Warner  (Warner  and  others, 
2005)  to  get  clarification  on  certain  aspects  of  the  model  and  its  implementation.  Finally,  we 
completed  our  analysis  and  developed  our  recommendations. 

1.5.  Report  Overview. 

In  Chapter  2  of  this  report,  we  begin  with  a  discussion  of  the  overall  concept  of  STC  and 
how  the  commissioning  mission  model  is  implemented.  In  Chapter  3  we  provide  details  about 
problems  we  saw  with  the  current  implementation.  In  Chapter  4,  we  will  suggest  a  simplified 
structure  for  model  implementation  in  Microsoft  Excel,  followed  by  detailed  instructions  on  how 
to  use  Excel’s  built-in  functions  to  handle  the  various  calculations  and  data  referencing.  In  this 
chapter  we  will  also  provide  suggestions  on  some  other  modifications  and  additions  that  can  be 
made,  with  an  emphasis  on  an  alternative  minimum  mission  value.  In  Chapter  5,  we  will  discuss 
the  use  of  Data  Envelopment  Analysis  to  measure  efficiency,  and  what  data  requirements  may  be 
needed  to  utilize  this  tool.  Our  conclusion  is  given  in  Chapter  6. 
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Chapter  2:  STC  Model  and  Current  Implementation. 

2.1.  Overall  Concept. 

An  initial  recruitable  market  size  is  estimated  using  data  on  the  number  of  baccalaureate 
degrees  conferred  using  current  data  from  the  National  Center  for  Education  Statistics  (NCES). 
That  number  is  then  adjusted  by  subtracting  ineligible  students  (i.e.  non-U.S.  citizens,  part-time 
students,  or  students  older  than  27  years  of  age)  and  adding  students  that  join  the  ROTC  program 
through  campus  partnerships  .  A  final  6%  reduction  is  applied  if  sister  service  ROTC  programs 
are  present  at  the  same  institution.  The  result  of  these  adjustments  is  called  the  Adjusted 
Recruitable  Market.  The  Adjust  Prime  Market  (the  market  of  interest)  results  from  the  product 
of  the  Adjusted  Recruitable  Market,  the  percentage  of  First  Stringers,  and  the  percentage  of 
SALs.  The  Adjusted  Mission  is  the  product  of  the  Adjusted  Prime  Market  and  a  market 
penetration  factor  (expressed  as  a  percentage),  plus  any  applicable  partnership  cadets  (see 
footnote  2).  The  Adjusted  Mission  represents  the  overall  commission  mission  for  a  particular 
college  or  university. 

2.2.  Implementation. 

STC  is  implemented  in  two  separate  Microsoft  Excel  spreadsheets.  The  bulk  of  the 
information  on  a  particular  school  (including  its  number  of  baccalaureate-level  graduates)  and 
the  relevant  numbers  from  the  OCMPS  are  stored  in  one  file,  while  the  process  for  adjusting  the 
recruitable  market  size  is  carried  out  in  a  separate  spreadsheet. 


Working  DST 

DST 

Worksheet 


I 

I 


Figure  1:  STC  Spreadsheets 

2  If  the  partnership  is  with  a  school  less  than  45  minutes  away  and  the  distant  school  contributes  at  least  25%  of  a 
program’s  cadets,  a  25%  increase  in  applied  to  the  Adjusted  Recruitable  Market.  If  a  program  does  not  meet  both 
criteria,  any  cadets  resulting  from  a  partnership  are  added  directly  into  the  Adjusted  Mission. 
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The  arrows  in  the  diagram  in  Figure  1  do  not  represent  any  actual  transfer  of  data 
between  the  two  files  (there  are  no  links  for  this  action  built  into  either  file).  It  only  indicates 
that  data  is  transcribed  by  the  user  from  one  file  to  another.  The  “Working  Model”  worksheet  is 
used  to  organize  the  data  from  the  OCMPS  and  to  ‘wargame’  values  for  commissioning  mission 
and  market  penetration  percentage  based  on  school  type.  Figure  2  shows  the  entire 
implementation  starting  from  the  Recruitable  Market  (data  from  NCES)  and  ending  with  the 
Adjusted  Mission. 


Figure  2:  STC  Model,  Overall  Commission  Mission 
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Chapter  3:  Problems  with  Current  Implementation. 

In  general,  very  little  actual  calculation  takes  place  within  the  model  as  implemented  in 
the  spreadsheets.  Much  of  the  data  appears  to  be  transferred  manually  (using  a  copy-and-paste 
operation)  as  opposed  to  direct  cell  addressing.  The  only  calculation  leading  up  to  the  figure  for 
Adjusted  Mission  that  is  done  consistently  for  all  272  schools  is  the  Prime  Market  (shown  in  red 
on  the  left  side  of  Figure  2).  However,  this  adds  no  value  to  the  outcome  since  no  other  process 
or  calculation  in  the  spreadsheet  depends  upon  this  figure  (the  prime  market  calculation  only 
becomes  meaningful  when  it  is  calculated  from  the  Adjusted  Recruitable  Market).  A  column  for 
Base  Mission  appears  in  the  School  Data  worksheets,  but  has  been  transcribed  manually  from  the 
Working  Model  worksheet  (and  actually  does  not  appear  to  reflect  the  values  for  Base  Mission 
calculated  in  the  “Working  Model”  worksheet),  and  is  also  not  used  anywhere  else  in  the 
spreadsheet.  Other  problems  in  the  current  implementation  are  discussed  below. 

3.1.  Required  Market  Penetration  in  the  “Working  Model” 
Worksheet. 

The  left-hand  side  of  this  table  is  used  to  determine  a  base  value  for  required  market 
penetration  and  a  basic  mission  value  for  each  school  type  (see  Figure  3).  The  “Basic  Mission 
by  School 


BASE  MODEL 


- 4 

- 

- * 

- 5 

- * 

School  Type 

Prime  Market 

Avg  Prime 
Market  per 
School 

Percent  ol 

Total  Prime 
Market 

#  Schools  in 
Category 

Aggregate 
School  Type 
Prod 

Basic  mission 
by  School 

Type 

Required  Mkt 
Pentration 

Meqa-Hiqh 

6222 

415 

19.1  X 

15 

873 

58 

14.05* 

Mega-Prestigious 

1918 

384 

5.9X 

5 

269 

54 

14.05* 

Large-High 

4492 

184 

13.65* 

27 

622 

23 

W.05* 

Large-Moderate 

5430 

118 

16.65* 

46 

762 

17 

W.05* 

|  Large-Prestigious 

932 

155 

2.95* 

6 

131 

22 

W.05* 

-  - 

Large-Standard 

1777 

99 

5.45* 

18 

249 

14 

W.05* 

Medium-Hiqh 

823 

75 

2.55* 

11 

115 

10 

W.05* 

Medium-Moderate 

2299 

77 

7.05* 

30 

322 

11 

W.05* 

Medrum-Prestiqious 

1376 

92 

4.25* 

15 

193 

13 

MJOK 

Medium-Standard 

1486 

68 

4.65* 

22 

208 

9 

W.05* 

Smai-Hkjh 

77;: 

45 

2454 

17 

108 

6 

W.05* 

Smal-Moderate 

348 

32 

115* 

11 

49 

4 

W.05* 

Smal-Presligious 

99 

33 

0  35* 

3 

14 

5 

W.05* 

Small-Standard 

409 

34 

135* 

12 

57 

5 

W.05* 

HBCUs 

797 

35 

2.45* 

23 

112 

5 

W.05* 

Senior  Military  College 

2298 

383 

7.05* 

6 

322 

54 

14.05* 

Military  Junior  College 

1200 

^24^^ 

3.75* 

5 

168 

34 

14.05* 

Totals 

32619 

1005* 

272 

4575 

14  OX 

Figure  3:  Required  Market  Penetration  calculation  in  " Working  Model". 
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'I 

Type”  provides  a  useful  starting  point  for  missions  based  on  the  OCMPS  data  .  These  values  are 
adjusted  in  the  “Wargaming”  section  to  account  for  known  outlier  school  types.  However,  the 
“Required  Market  Penetration”  values  are  not  actually  based  on  the  data  stored  in  each  row,  but 
merely  reflect  the  ratio  of  Commission  Mission  (4575)  to  the  total  Prime  Market  size  (32619),  so 
that  4575/32619=0.14  (these  values  are  found  at  the  bottom  of  the  “Aggregate  School  Type 
Prod”  and  “Prime  Market”  columns,  respectively).  This  is  because  the  value  for  Prime  Market 


used  in  the  by-row  calculations  cancels  out  (see  Equation  1). 

AggregateSchoolType  Prod 


Required  Mkt  Penetration  = 


Prime  Market 

(Cmsn  Msn)(Percent  of  T otal  Prime  Market) 


(CmsnMsn) 


Prime  Market 

Prime  Market 
Prime  Market  T otal 


Prime  Market 
CmsnMsn  4575 


=  0.14 


Prime  Market  Total  32619 
Equation  1:  Result  of  current  calculation  for  Required  Market  Penetration  by  school  type. 


There  is  no  real  error  in  doing  the  calculation  for  Required  Market  Penetration  in  this 
manner  -  we  merely  point  out  that  the  structure  of  the  worksheet  obscures  the  source  of  the 
value,  and  does  not  really  reflect  the  fact  that  it  is  merely  calculating  a  value  for  required  market 
penetration  that  is  aggregated  across  the  entire  body  of  schools. 

3.2.  Underestimation  of  the  Adjusted  Prime  Market. 

There  are  three  sources  for  this  underestimation.  The  first  source  is  an  accepted 
limitation  of  the  model  -  that  of  only  accounting  for  the  percentage  of  First  Stringers,  and 
excluding  other  acceptable  groups  (i.e.  Second  Stringers).  The  second  source  is  not  accounting 
for  the  possibility  that  a  student  might  fall  into  more  than  one  category  among  the  filters 
(students  who  are  not  U.S.  citizens,  are  part-time,  or  are  older  than  27).  It  is  possible,  however, 
that  this  overlap  is  negligible.  The  last  is  determining  the  number  of  part-time  students  by 
calculating  the  ratio  of  part-time  to  full-time  students  (this  calculation  is  performed  in  the 
Working  DST  spreadsheet).  These  two  groups  are  mutually  exclusive,  and  the  resulting  ratio 

3  However,  it  should  be  noted  that  neither  the  Base  Mission  or  Adjusted  Mission  values  that  appear  in  the  “Working 
Model”  worksheet  appear  to  transfer  to  the  “School  Data”  worksheets  intact  -  new  values  appear  in  the  respective 
columns  on  the  “School  Data”  worksheets,  implying  further  adjustment. 
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underestimates  the  proportion  of  full-time  students.  Dividing  the  number  of  full-time  students  by 
the  school’s  total  enrollment  gives  the  proper  proportion,  which  can  be  used  in  determining  the 
estimated  size  of  the  prime  market  (this  is  shown  in  Figure  6  and  discussed  in  more  detail  in 
section  4.2.3). 

3.3.  Inconsistent  transition  from  DST  back  to  School  Data 
worksheet. 

The  first  really  significant  input  on  the  School  Data  worksheets  is  the  Adjusted 
Recruitable  Market,  which  results  from  the  Working  DST  calculations.  The  Adjusted  Prime 
Market  is  calculated  from  it  using  the  OCMPS  percentages.  This  is  where  the  transition 
inconsistency  exists:  it  is  unclear  where  the  calculation  is  performed.  The  Working  DST  does 
not  have  the  same  complete  list  of  all  272  schools  -  it  appears  to  be  more  of  a  tool  that  is 
intended  to  handle  one  input  at  a  time  and  only  produces  one  output.  In  the  cases  where  the 
Adjusted  Prime  Market  figure  is  entered  as  a  constant  in  the  School  Data  worksheets  (see  Figure 
4),  it  is  assumed  that  the  number  was  produced  using  the  Working  DST.  However,  26  schools 
have  their  Adjusted  Prime  Market  value  calculated  in  the  School  Data  worksheet  (see  Figure  5), 
using  values  entered  in  the  worksheet  itself. 

•  i  -w  ^  l  0V  —I  'V  I  I  SO  «.  ▼  AT  I  IS 


Figure  4:  Constant  Value  for  Adjusted  Prime 
Market 

3.4.  Incorrect  OCMPS  values. 

In  13  cases,  the  wrong  OCMPS  percentages  are  entered  for  that  school.  Closer 
examination  revealed  that  the  percentages  entered  were  based  on  the  school  size  that  resulted 
from  the  ‘BA  Grads’  value  (see  Figure  2)  value  instead  of  the  Adjusted  Recruitable  Market 
value. 
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^||4e  Prime  Adjuste 

Mkt  Mkt  Msn 

CAROLINA  UNIVERSITY 

Larqe-Moderate 

6 

213511  1211) 

BETH  CITY  STATE  UNIV 

Small-Standard 

4 

1521  8T 

"TON  UNIVERSITY 

Small-High 

7 

747  55 

OLK  STATE  UNIVERSITY 

Small-Standard 

6 

723 1  44 1 

Figure  5:  Function  for  Adjusted  Prime  Market 


3.5.  Duplicate  Data. 

There  are  two  instances  where  the  same  data  is  repeated  in  more  than  one  place  within 
the  ‘CC  STC  Working  Base  Model’  spreadsheet.  ‘BA  Grads’  (appears  in  both  the  school  type 
group  worksheets  and  the  School  Data  worksheets)  and  ‘Base  Mission  by  School  Type’  (appears 
in  both  the  Working  Model  and  School  Data  worksheets)4. 


4  This  data  is  not  technically  duplicated,  since  the  values  do  no  match.  It  appears  that  the  values  in  the  School  Data 
worksheets  have  been  adjusted  from  those  shown  on  the  Working  Model  worksheet.  However,  the  data  field  is 
duplicated. 


9 


Chapter  4:  Recommendations. 

4.1.  Overview  of  recommended  changes. 

In  general  terms,  we  recommend  replacing  the  current  implementation  of  two  separate 
Excel  spreadsheet  files5  with  a  single  Excel  spreadsheet  file  containing  four  worksheets,  and  to 
restrict  each  workbook  to  a  single  academic  year  (using  a  filename  that  reflects  the  AY  data 
contained  in  that  file).  The  primary  purpose  of  this  change  is  to  eliminate  duplicate  data  within 
the  files,  reduce  or  eliminate  errors  due  to  manual  data  entry,  and  to  eliminate  the  transition 
between  the  School  Data  worksheets  and  the  ‘Working  DST’  file  altogether.  The  14  worksheets 
in  the  “CC  STC  Working  Base  Model”  file  should  be  removed  because  they  do  not  perform  any 
unique  function  in  the  file  (the  summarizations  performed  in  them  can  be  handled  by  employing 
appropriate  functions  on  a  separate  summary  worksheet  -  this  will  be  discussed  in  section  4.1.4 
and  4.2.4).  Additionally,  this  restructuring  will  result  in  a  more  modular  arrangement  by  keeping 
data  and  reference  tables  separated  from  data  processing  and  data  summaries.  The  four 
worksheets  can  be  thought  of  as  modules  that  perform  the  following  functions: 

4.1.1.  School  Data  Module  (SDM). 

This  module  is  used  for  data  storage  only  and  will  contain  all  the  data  for  a  school  on  (at 
a  minimum)  annual  baccalaureate-level  graduates,  total  enrollment,  full-time  enrollment,  the 
number  of  non-US  citizens,  number  of  students  over  the  age  of  27,  partnership  data,  and 
information  on  sister  service  competition.  Most  data  in  this  module  is  either  text  or  numeric,  but 
the  use  of  a  couple  of  indicator  (or  Boolean  -  two-state  -  variables)  is  recommended.  This  also 
simplifies  the  data  entry  process,  since  all  data  pertaining  to  a  particular  school  is  contained  in  a 
single  worksheet. 

4.1.2.  OCMPS  Data  Module  (ODM). 

This  module  contains  a  table  with  the  figures  for  “Percent  SALs”  and  “Percent  First 
Stringers”  assigned  to  their  respective  school  type  (from  the  OCMPS  results).  It  is  additionally 
recommended  that  the  boundary  values  for  academic  reputation  and  school  size  are  stored  here. 
This  will  facilitate  the  ability  to  automatically  look  up  a  school’s  values  for  “Percent  SALs”, 
“Percent  First  Stringers”,  and  “Required  Market  Penetration”  based  solely  on  data  stored  in  the 

5  The  file  ‘CC  STC  Working  Base  Model’  has  19  worksheets  (the  ‘Main’  and  “Definitions’  worksheets  were  not 
included),  and  the  ‘Working  DST’  file  has  one  worksheet. 
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School  Data  Module.  This  module  would  likely  need  modifications  to  its  contents  as  the 
OCMPS  progresses,  and  could  be  eliminated  altogether  once  a  percentage  for  SALs  and  First 
Stringers  is  determined  for  each  school. 

4.1.3.  Mission  Data  Processing  Module  (MDPM). 

This  module  is  responsible  for  retrieving  school  data  and  performing  the  necessary 
calculations  to  determine  a  value  for  a  commissioning  mission  for  each  school.  It  is  where  all 
the  core  functions  of  the  model  are  executed.  All  columns  in  this  module  contain  functions  that 
either  perform  calculations  on  data  stored  in  other  modules  (or  workbooks),  look  up  data  stored 
in  other  modules  (or  workbooks),  or  some  combination  of  these  functions.  The  only  column 
intended  to  store  raw  numerical  data  is  a  provision  that  allows  and  tracks  deviations  from  the 
model’s  output.  This  aspect  will  be  discussed  in  section  4.4. 1 . 

4.1.4.  Data  Summary  Module  (SUM). 

Any  summarized  data,  such  as  the  total  number  of  schools  within  a  region  or  brigade, 
other  categorical  totals,  or  grand  totals,  be  stored  here  -  separate  from  the  raw  data  or  other 
modules.  There  are  a  number  of  useful  Excel  functions  that  can  be  used  to  summarize  data  from 
anyplace  in  the  workbook  (or  even  data  external  to  it,  if  needed). 

4.1.5.  Historical  Data  Module  (HIST). 

This  can  essentially  be  the  same  as  the  NINESTEP  format  currently  in  use.  It  is  not 
strictly  necessary  to  include  this  data  as  a  module  within  the  same  workbook  as  the  SDM,  ODM, 
MDPM,  and  SUM  modules.  In  fact,  the  preferred  method  would  be  to  keep  historical  data  in  a 
separate  workbook  and  simply  refer  to  the  appropriate  cells  as  needed.  The  primary  purpose  for 
this  module  with  respect  to  the  calculation  of  the  commissioning  mission  is  to  facilitate  the 
implementation  of  the  Alternate  Minimum  Mission  Threshold  that  will  be  discussed  in  Section 

4.4.2. 

4.2.  A  Detailed  Look  at  the  Changes. 

Figure  6  shows  a  diagram  that  represents  our  recommended  implementation.  The  SDM 
is  on  top,  and  contains  all  the  numeric  data  and  indicator  variables  pertaining  to  each  school.  No 
functions  are  necessary  or  recommended  on  this  worksheet.  Note  that  the  arrows  pointing  from 
the  SDM  to  items  in  the  MDPM  represent  functions  in  the  MDPM  that  contain  references  back 
to  the  relevant  cell  in  the  SDM.  The  ODM  contains  the  data  shown  in  the  diagram,  but  can  also 
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contain  the  boundary  values  for  school  size  and  academic  reputation  that  define  a  particular 
school  type.  The  correct  entry  in  the  table  can  be  accessed  with  the  Excel’s  lookup  functions. 
Finally,  the  MDPM  performs  all  the  actual  calculations  needed  to  deliver  a  value  for 
commissioning  mission. 


Figure  6:  Proposed  Model  Implementation 

4.2.1.  Structure  of  the  SDM. 

As  we  stated  before,  the  SDM  should  contain  (at  a  minimum)  columns  for  the  data  shown 
in  Figure  6.  It,  of  course,  also  may  contain  any  other  data  that  relates  to  a  particular  school  (i.e. 
FICE,  BDE,  city,  state,  ZIP  code,  etc.).  The  primary  function  of  this  module  is  raw  data  storage. 
We  recommend  that  any  information  that  is  derived  from  the  raw  data  (i.e.  school  type  from  the 
OCMPS,  such  as  “Small- Standard”)  be  reported  in  the  MDPM  using  the  appropriate  function. 
The  order  of  the  data,  however,  is  not  important  -  any  arrangement  of  the  columns  that  makes 
data  entry  and  review  easier  will  work. 
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4.2.2.  Structure  of  the  ODM. 

Our  recommended  structure  for  this  module  actually  includes  three  tables:  one  table  that 
reflects  the  academic  reputation  values,  one  that  reflects  the  school  size  values,  and  one  that 
assigns  the  SAL  and  First  Stringer  percentages  to  their  respective  school  types.  It  can  be  laid  out 
as  shown  in  Figure  7. 


A 

B 

C 

D 

E 

F 

1 

School  Size  Table 

2 

Lower  Bound 

Size 

Description 

3 

0 

Smal 

(defned  as  less  than  300  undergraduate  degrees) 

4 

soo 

Medium 

[defned  as  between  300  and  1499  undergraduate  degrees) 

5 

1500 

Large 

[defned  as  between  1500  and  2999  undergraduate  degrees) 

6 

3000 

Mega 

[defned  as  3000'  undergraduate  degrees  or  more) 

7 

8 

Academic  Reputation  Table 

9 
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10 
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Standard 

11 

2.5 

Moderate 

12 

3.2 

High 

13 
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Prestigious 

14 

15 

OCMPS  Cluster  Table 

Required 
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%  1st  Stringers 

%  SALs 

Market 

16 

Penetration 

17 

Smal- Standard 

134% 

40.0% 

13.1% 

IS 
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13.9% 

51.0% 

19.2% 

19 

Smafr  Moderate 

135% 

40.0% 

13.1% 

20 

Smal- High 

14.6% 

50.0% 

12.3% 

21 

Mega- Prestigious 

15.3% 

52.0% 

12.9% 

22 

Mega- High 

17.5% 

53.0% 

12.9% 

23 

Medium- Standard 

13.5% 

45.0% 

12.3% 

24 

Medium- Prestigious 

14.9% 

53.0% 

129% 

25 

Medium- Moderate 
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12.9% 

26 

Medium- High 

15.6% 

47.0% 

12.9% 

27 
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13.2% 

40.0% 
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28 

Large- Prestigious 

13.3% 

49.0% 

12.9% 

29 

Large- Moderate 

13.5% 

42.0% 

129% 

30 

Large- High 

14.3% 

45.0% 

12.9% 

31 

MJC 

50  0% 

40.0% 

15.6% 

32 
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50  0% 

40.0% 
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Figure  7:  OCMPS  Data  Module 


The  School  Size  table  and  the  Academic  Reputation  Table  are  designed  to  be  referenced 
using  Excel’s  LOOKUP  function,  and  the  OCMPS  Cluster  Table  is  designed  to  be  referenced 
using  the  VLOOKUP  function.  The  advantage  of  this  approach  is  that  it  will  determine  the 
appropriate  school  type,  along  with  the  figures  for  ‘Percent  First  Stringers’,  ‘Percent  SALs’,  and 
‘Required  Market  Penetration’  automatically,  and  any  changes  in  the  values  stored  in  the  ODM 
will  be  automatically  reflected  in  the  MDPM. 

4.2.2. 1 .  Getting  School  Type. 

The  School  Size  Table  and  Academic  Reputation  Table  are  referenced  first  in  order  to 
produce  the  appropriate  school  type  label.  This  can  be  done  in  one  step  using  a  compound 
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function  which  utilizes  the  IF,  LOOKUP,  and  CONCATENATE  functions.6  In  the  example 
shown  in  Figure  8,  we  see  that  the  school  size  is  determined  from  taking  the  value  in  cell  G2  (for 
the  first  school  in  the  list)  and  using  the  LOOKUP  function  to  retrieve  the  right  category  from 
the  School  Size  Table  in  the  ODM.  Likewise,  the  school  reputation  is  determined  by  taking  the 
value  for  the  school  stored  in  the  SDM  and  using  LOOKUP  to  retrieve  the  correct  category  from 
the  Academic  Reputation  Table.  The  result  of  the  compound  function  then  becomes  the  entry 
argument  when  referencing  the  OCMPS  Cluster  table. 
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Figure  8:  Getting  School  Type  from  the  tables  in  the  ODM. 

4. 2.2. 2.  Getting  OCMPS  Percentages. 

Once  a  school  type  has  been  determined,  the  OCMPS  percentages  (Percent  First  Stringers 
and  Percent  SALs)  for  that  type  can  be  found  using  the  VLOOKUP  function.  An  example  that 
finds  “%  First  Stringers”  is  shown  in  Figure  9. 


6  The  function  first  checks  to  see  if  the  school  is  a  MJC  or  SMC.  If  not,  then  the  two  LOOKUP  commands  locate 
the  appropriate  size  and  reputation,  respectively,  and  the  CONCATENATE  function  places  them  in  a  single  string, 
joined  with  a  hyphen. 
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Figure  9:  Getting  OCMPS  Percentages  from  the  ODM 


Here,  the  type  “Small-Standard”  becomes  the  first  argument  (the  lookup  value)  in  the 
YLOOKUP  function.  The  function  tries  to  match  the  lookup  value  with  an  entry  appearing  in 
the  left-hand  column  of  the  table  (which  is  column  1).  If  a  match  is  found,  it  returns  the  value 
stored  in  a  cell  whose  row  is  the  same  as  the  lookup  value  and  whose  column  is  identified  in  the 
third  argument  of  the  VLOOKUP  function  (2,  in  this  case).  The  values  for  “%  SALs”  and 
“Required  Market  Penetration”  can  be  found  similarly  -  the  only  change  is  the  third  argument  in 
the  VLOOKUP  function  (which  would  change  to  3  and  4,  respectively).  The  “Required  Market 
Penetration”  percentages  shown  in  Figure  9  were  taken  directly  from  the  wargaming  section  of 
the  “Working  Model”  worksheet  in  “CC  STC  Working  Base  Model”  spreadsheet.  They  are 
applied  in  the  Mission  calculation  without  modification.  This  ensures  a  consistent,  systematic 
approach  to  determining  the  commissioning  mission  values.  Deviations  from  these  values  (via 
negotiation)  are  accounted  for  using  one  of  the  methods  outlined  in  Section  4.4.1. 
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4.2.3.  Structure  of  the  MDPM. 

The  primary  function  of  this  module  is  to  produce  the  commissioning  mission  values. 
There  should  be  no  need  to  enter  or  store  any  raw  data  here  -  all  cells  on  this  worksheet  should 
either  contain  direct  references  to  cells  on  other  worksheets  (when  data  is  “being  brought 
forward”  with  no  calculations  performed,  such  as  a  school’s  name),  or  perform  a  needed 
calculation  or  other  process  on  the  data.  An  example  of  a  basic  layout  that  reflects  the  structure 
shown  in  Figure  6  is  now  shown  in  Figure  10.  Cells  in  the  “School  Type”  column  use  the 
compound  function  shown  in  Figure  8  and  discussed  in  section  4.2.2. 1.  Cells  in  the  “%  First 
Stringers”,  “%  SALs”,  and  “Required  Market  Penetration”  columns  use  the  function  shown  in 
Figure  9  and  discussed  in  section  4.2. 2.2.  The  order  of  the  columns  in  this  example  reflects  the 
process  shown  in  Figure  6,  where  we  adjust  BA  Grads  to  get  the  Recruitable  Market  and  use  that 
value  to  find  the  right  School  Type.  We  use  School  Type  to  find  the  right  OCMPS  percentages, 
which  are  then  used  in  the  calculations  for  Prime  Market  Estimate  and  Mission. 
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Figure  10:  An  example  MDPM.  Note  that  the  school  name  is  linked  over  from  the  SDM. 


4.2.3. 1 .  Getting  the  Recruitable  Market  Estimate. 

Functions  stored  in  columns  E,  F  and  G  perform  the  calculations  necessary  to  generate  an 
estimate  for  “Recruitable  Market”  (the  equations  for  each  step  are  shown  in  Figure  6).  The 
progression  on  the  worksheet  is  left-to-right.  All  of  the  data  used  for  the  “Adjustment  1” 
(Ineligible  Student  Filter)  calculation  is  referenced  from  the  SDM.  This  adjustment  results  in  an 
interim  value  ( Gu )  that  is  no  greater  than  the  BA  Grads  value.  The  ROUND  function  is 

employed  to  produce  an  integer  value  (see  Figure  11).  Notice  also  that  all  ineligible  students 
(part-time,  overage,  non-citizens)  are  removed  from  the  BA  Grads  value  in  a  single  calculation. 
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The  calculation  for  “Adjustment  2”  (Partnership  Adjustment)  uses  the  previous  result 
( G  ,  ( )  and  additional  data  from  the  SDM  to  produce  another  interim  result,  G A2 .  This  value  will 


always  be  no  less  than  G  u  .  Note  also  that  this  function  employs  one  of  our  Boolean  variables 


(Pr  which  is  1  if  a  partnership  school  is  within  45  minutes  of  the  program  school  and  contributes 
at  least  25%  of  the  programs  enrolled  cadets,  and  0  otherwise).  If  the  value  is  zero,  GA2  =  GAX  ■ 
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The  final  adjustment  to  BA  Grads  is  Adjustment  3  (Sister  Service  Adjustment)  which 
results  in  the  Recruitable  Market  Estimate,  R.  The  adjustment  employs  our  other  Boolean 
variable,  S,  which  is  1  if  another  ROTC  program  exists  on  campus,  and  0  otherwise.  It  has  the 
same  effect  on  this  calculation  as  P  did  in  Adjustment  2.  The  value  for  R  will  never  be  greater 


than  GA2. 


Figure  13:  Calculating  Adjustment  3.  This  result  is  the  Recruitable  Market  Estimate 


4. 2. 3. 2.  Getting  the  Prime  Market  Estimate. 

The  “Prime  Market  Estimate”  calculation  is  just  the  product  of  the  values  in  columns  G, 

I,  and  J.  All  of  these  values  are  already  present  in  the  MDPM,  so  there  are  no  references  to  other 
worksheets  in  the  formula. 
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This  is  the  only  time  in  our  implementation  that  an  estimate  of  the  Prime  Market  is  derived. 

4.2. 3. 3.  Getting  the  Commissioning  Mission. 

The  value  for  “Mission”,  as  shown  in  the  formula  in  Figure  6,  is  the  product  of  the  Prime 
Market  Estimate  and  the  Required  Market  Penetration,  plus  any  cadets  resulting  from  a 
partnership  school  outside  the  45-minute  radius.  An  example  of  the  final  calculation  is  show  in 
Figure  15. 
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Figure  15:  Calculating  the  Commissioning  Mission. 


4.2.4.  Suggested  Functions  for  SUM. 

There  are  several  useful  functions  that  can  be  employed  in  the  SUM  module  to  provide 
summary  information  on  the  data  found  in  other  modules.  Some  of  the  most  commonly  used 
functions  include  COUNT,  COUNTIF,  and  SUMIF  (consult  the  Excel  Help  files  for  details  on 
their  use).  By  way  of  example,  we  will  show  a  simple  table  that  provides  counts  of  the  different 
school  types  in  a  compact  format.  In  this  case  we  will  use  both  the  COUNTIF  and 
CONCATENTATE  functions.  In  Figure  16,  we  show  how  a  compound  function  using  these 
two  built-in  functions  can  be  used  to  provide  a  count  of  the  number  of  schools  in  each  category. 
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The  syntax  of  the  COUNTIF  function  requires  a  range  of  cells  in  the  first  argument,  and  the 
counting  criterion  in  the  second  argument.  It  is  in  the  second  argument  that  we  use  the 
CONCATENATE  function. 
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Figure  16:  An  example  of  a  school  type  summary. 


The  first  argument  in  the  COUNTIF  function  refers  to  the  column  on  the  MDMP  that 
contains  the  school  types  (size-reputation).  In  cell  B2,  we  have  built  the  criterion  by  using  the 
CONCATENATE  function  to  create  a  string  from  the  contents  of  cells  B1  and  A2,  separated  by 
a  hyphen  (the  result  of  which  is  “Small- Standard”).  It  then  returns  the  number  of  schools  that 
have  “Small-Standard”  stored  in  the  school  type  column.  The  dollar  signs  used  in  the 
CONCATENATE  terms  ensure  that  the  function  continues  to  reference  the  cells  where  the 
labels  are  stored  after  it  is  copied  to  the  other  15  summary  cells.7 

4.3.  A  Word  about  Inputs. 

We  have  seen  no  indications  of  any  problems  with  the  model’s  current  inputs.  Since  the 
OCPMS  is  ongoing,  the  percentages  of  First  Stringers  and  SALs  will  eventually  be  derived  from 
survey  data  at  the  school  level  and  will  no  longer  need  to  be  aggregated  across  school  types.  As 
for  propensity,  when  the  First  Stringer  criterion  is  compared  with  measures  of  propensity  in  the 
DoD’s  Youth  Attitude  Tracking  Study  (YATS),  the  approaches  are  similar  (1995  Annual 
Defense  Report:  Appendix  G).  YATS  usually  asks  respondents  (who  are  still  in  high  school) 
directly  about  their  intention  to  enlist  (unless  they  provide  an  “unaided  mention”  of  intended 
military  service  (Orvis,  2001)),  which  is  essentially  a  direct  question  about  their  plans  for  the 
future  (something  that  senior  highs  school  students  tend  to  think  about).  This  direct  approach  is 
unlikely  to  have  the  same  effect  on  a  first  or  second  year  college  student,  since  at  that  point  they 
“have  already  embarked”  on  their  future  plans.  Because  of  this,  we  think  that  the  OCMPS  takes 

7  Please  note  that  the  SUM  label  that  appears  in  the  CONCATENATE  terms  refers  to  the  name  of  the  worksheet 
that  the  summary  table  is  stored  on,  and  not  Excel’s  built-in  SUM  function. 
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a  reasonable  approach  for  determining  propensity  by  determining  whose  attitudes  and  interests 
align  with  the  activities  of  Army  ROTC  and  Army  service. 

4.4.  Additional  Modifications. 

There  are  a  couple  of  other  tools  that  could  be  added  to  this  basic  implementation. 
Section  4.4. 1  is  a  modification  to  an  existing  calculation  in  the  current  STC  implementation. 
Primary  emphasis  is  this  section  is  given  to  the  Alternate  Minimum  Mission  Threshold  in 
Section  4.4.2. 

4.4.1.  Delta  to  Track  Mission  Changes. 

We  expect  model  fidelity  to  increase  as  more  data  from  the  ongoing  OCMPS  is 
introduced  into  the  model  (as  well  as  other  future  changes  discussed  in  Chapter  4).  However,  a 
method  is  needed  to  allow  changes  to  the  commissioning  mission  values  without  disturbing  the 
functions  stored  in  existing  cells  (these  changes  are  the  result  of  the  current  policy  discussed  in 
section  1 .4).  The  simplest  method  of  doing  this  would  be  to  add  two  columns  to  the  right  of  the 
“Mission”  column  in  the  MDPM:  the  first  column  represents  the  “Delta”  (the  increase  or 
decrease  in  the  “Mission”  value),  and  a  second  column  called  “Adjusted  Mission”,  which  is  just 
the  sum  of  the  values  for  “Mission”  and  “Delta”.  An  example  is  shown  in  Figure  17. 


M 

N 

0 

Mission 

□eta 

Adjusted 

Mission 

:  12  . 

j 

|=M2+N2 

12 

Figure  17:  Using  a  delta  to  adjust  "Mission"  values. 

This  also  allows  the  changes  in  the  “Mission”  values  to  be  explicitly  tracked  and  stored 
without  overwriting  the  previous  equations.  The  “Adjusted  Mission”  could  also  be  calculated  by 
changing  the  “Required  Market  Penetration”.  If  this  approach  is  preferred,  it  is  recommended 
that  the  “Delta”  column  (which  could  be  renamed  “New  Required  Market  Penetration”)  contain 
a  function  that  produces  the  required  percentage  value,  and  “Adjusted  Mission”  would  contain 
constants.  If  we  let  np%  equal  “Required  New  Market  Penetration”,  then 
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Adjusted  Market  -  P2 
M 


=  nP% 

Equation  2:  Equation  for  expressing  delta  as  a  new  required  market  penetration  percentage. 

4.4.2.  Alternate  Minimum  Mission  Threshold  Derived  from  Production 
History. 

In  the  current  STC  implementation,  if  the  model  produces  a  commissioning  mission 
value  less  than  six,  the  value  is  increased  to  six.  USACC  has  decided  that  this  should  be  the 
absolute  minimum  commissioning  mission  assigned  to  any  school.  Since  the  overall 
commissioning  mission  is  increasing  (3900  to  4400),  we  examined  historical  production  data  that 
we  were  provided  for  insights  on  any  other  potential  minimum  mission  assignments.  The  goal  of 
this  portion  of  the  analysis  was  to  increase  the  chances  of  achieving  the  new  overall 
commissioning  mission  of  4400  officers. 

4.4.2. 1.  Initial  Observations. 

Production  data  for  1991-2004  was  examined.  The  1990  data  was  excluded  because  the 
overall  commissioning  mission  was  significantly  higher  that  year  than  in  subsequent  years.  The 
1991-2004  data  for  each  school  was  tested  for  normality  using  a  Ryan- Joiner  test.  This  test  is 
similar  to  using  a  normal  probability  plot  to  determine  whether  observed  data  could  have  been 
taken  from  a  normal  population  (an  approximation  in  this  case  because  our  data  is  actually 
discrete,  whereas  the  normal  distribution  is  continuous).  However,  the  Ryan-Joiner  test  is  a 
quantitative  measure,  which  allowed  us  to  test  all  the  schools  at  once.8  Under  this  test,  the  null 
hypothesis  was  rejected  for  seven  schools  at  a  significance  level  of  a  =  0.01  (which 
corresponds  to  a  sample  correlation  coefficient  of  r  =  0.9049 ).  Of  those  seven  schools,  none 
had  a  sample  correlation  coefficient  lower  than  0.86.  Figure  18  shows  the  distribution  of  sample 
correlation  coefficients  for  270  of  the  schools  (two  schools  were  removed  due  to  insufficient 
data). 


8  For  an  explanation  of  the  Ryan-Joiner  test,  see  Appendix  A. 
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Sample  Correlation  Coefficients 


Figure  18:  Histogram  of  sample  correlation  coefficients  from  the  Ryan- Joiner  test. 

Since  the  sample  size  for  each  school  is  relatively  small,  we  concluded  that  the  normal 
distribution  is  a  good  approximation  for  the  data.  We  used  the  sample  mean  and  standard 
deviation  for  each  school  as  estimates  for  the  normal  distribution’s  parameters  [J  and  a. 

4.4.2. 2.  Methodology. 

A  simulation  was  conducted  that  allowed  each  school  to  generate  random  commissioning 
mission  values  from  the  normal  distributions  that  were  estimated  from  production  data. 

However,  a  limitation  was  imposed  on  each  distribution  as  to  where  those  random  numbers 
could  be  drawn  from.  The  point  here  was  to  determine  if  there  was  a  minimum  percentile, 
applied  uniformly  to  each  school’s  distribution,  which  would  result  in  a  total  production  of  4400 
most  of  the  time.  To  illustrate  this,  we’ll  examine  one  school,  Alabama  A&M  University.  This 
school  had  a  production  sample  mean  of  12.36  and  a  sample  standard  deviation  of  4.91.  If  we 
allow  the  simulation  to  generate  random  values  from  the  entire  normal  distribution,  the  vast 
majority  of  the  values  will  be  in  the  interval  (-2.37,  27.09),  and  about  2/3  of  the  values  will  be  in 
the  interval  (7.45,  17.27).  Figure  19  shows  the  distribution’s  pdf,  and  the  histogram  on  the  right 
shows  simulation  results  from  1000  samples. 
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Alabama  A&M  Production 


z:  P[-2.83  <=z<=  2.83]  =  1.0 
Raw:  P[-1 .5  <=X<=  26.2]  =  1.0 
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-1.5  26.2 


Mean:-.|~12.36  StdDev:  J 491 


Figure  19:  Simulating  school  production  from  the  whole  distribution. 


According  to  US  ACC  policy,  no  school  will  be  assigned  a  commissioning  mission  lower 
than  six.  If  we  set  this  as  the  lower  limit  for  Alabama  A&M,  we  get  the  following  results: 


Figure  20:  A  lower  threshold  of  6  is  imposed. 


This  doesn’t  appear  to  set  the  bar  very  high  for  Alabama  A&M,  since  the  graphs  in  Figure  20 
imply  that  they  would  have  no  problem  achieving  six  commissions. 

At  this  point,  it  is  appropriate  to  step  back  and  examine  the  overall  goal  once  more.  In 
the  current  model,  a  school  is  assigned  either  a  commissioning  mission  of  six,  or  the  mission 
value  produced  by  the  model,  whichever  is  greater.  This  rule  was  applied  as  the  only  constraint 
in  our  simulation,  so  that  a  school  was  allowed  to  generate  random  production  values  from  its 
distribution,  but  never  lower  than  six.  Using  this  rule,  the  simulation  almost  never  generated  a 
total  production  in  excess  of  4400  out  of  500  trials.  Based  on  this  result,  our  goal  was  to  find  an 
additional  constraint:  Assign  a  school  the  greatest  of  three  possibilities:  (1)  the  absolute 
minimum  of  six,  (2)  the  model  output,  or  (3)  the  nth  percentile  of  a  school’s  distribution.  The 
goal  was  to  find  a  value  for  n  (to  be  applied  to  all  schools)  such  that  the  total  production  was  at 
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least  4400  for  95%  or  more  of  the  simulation  trials.  In  our  simulation,  selecting  the  34th 
percentile  as  the  additional  constraint  produced  the  desired  result.  In  other  words,  if  all  schools 
in  the  simulation  generated  random  production  numbers  consistent  with  their  distributions,  but 
bounded  below  by  either  the  constant  6  or  the  distribution’s  34th  percentile  (whichever  was 
greater),  the  simulation  produced  a  total  of  4400  commissions  or  greater  at  least  95%  of  the  time. 
When  applied  to  Alabama  A&M,  we  get  the  results  shown  in  Figure  21. 


Alabama  A&M  Production 

Chart  Area- 1 


Figure  21:  The  lower  threshold  is  set  at  the  34th  percentile. 


In  this  case,  the  34th  percentile  is  greater  than  6,  so  it  would  be  used  as  the  minimum  for 
this  school.  If  the  model  output  for  Alabama  A&M  turns  out  to  be  greater  than  10  (rounding  the 
34th  percentile  to  the  nearest  value),  it  would  be  the  commissioning  mission  assigned. 

Otherwise,  10  would  be  the  assigned  mission.  For  more  detailed  information  on  the  simulation, 
see  Appendix  B. 

4.4.2. 3.  An  Important  Note  on  Assumptions. 

In  reality,  a  school  will  rarely  produce  the  exact  number  of  officers  in  a  given  year  as  the 
commission  mission.  It  was  assumed  that  increasing  the  mission  of  a  school  (at  least  by 
relatively  small  amounts)  would,  on  average,  result  in  a  production  increase.  This  is  important  to 
our  simulation  since  the  distributions  are  based  on  production  history,  but  we  are  using  the 
results  to  justify  a  change  in  mission  values.  So,  to  check  this,  we  performed  a  simple  regression 
on  a  subset  of  the  data  from  1991  to  2004.9  The  results  showed  that,  on  average,  an  increase  in 
mission  by  one  corresponded  to  an  increase  in  production  of  about  0.95  (this  value  was  the 


9  A  total  of  224  schools  were  examined.  Schools  were  excluded  from  this  analysis  because  they  either  had  missing 
data  or  were  known  outliers,  such  as  the  MJCs  and  SMCs. 
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average  slope  from  14  separate  regression  analyses;  the  standard  deviation  of  this  average  slope 
was  about  0.12).  This  implies  that  if  a  school’s  mission  increased  by  a  small  amount,  on  average 
it  will  meet  the  increase  in  mission. 

4.4.2.4.  Calculation  of  the  Percentile. 

For  this  new  criterion  to  be  included  in  the  mission  assignment  process,  the  34th 
percentile  needs  to  be  calculated  for  each  school.  There  are  two  ways  this  can  be  done,  but  the 
results  may  be  slightly  different  (the  magnitude  of  the  difference  is  related  to  how  closely  the 
data  was  approximated  by  the  normal  distribution;  in  our  examination  of  the  data,  220  of  the 
schools  showed  a  difference  of  one  or  less  when  the  two  methods  were  compared). 

4.4.2.4. 1 .  Directly  from  Historical  Data. 

A  percentile  can  be  calculated  directly  from  a  dataset  using  the  PERCENTILE  function 
in  Excel.  The  syntax  of  this  function  is  PERCENTILE(array,k),  where  ‘array’  is  a  cell  range 
that  contains  the  data,  and  ‘k’  is  the  percentile  expressed  as  a  probability  (between  0  and  1). 

4.4. 2.4. 2.  From  the  Approximate  Normal  Distribution. 

This  can  be  done  using  the  NORMINV  function.  The  syntax  for  this  function  is 
NORMINV(probability,mean,standard_dev),  where  ‘probability’  is  a  cumulative  probability 
value,  ‘mean’  is  the  arithmetic  mean  of  the  distribution,  and  ‘standard_dev’  is  the  standard 
deviation  of  the  distribution.  In  this  case,  our  cumulative  probability  value  is  0.34,  and  we 
substitute  the  sample  mean  and  sample  standard  deviation  from  each  school’s  production  data  as 
the  estimates  for  the  distribution  parameters.  The  function  returns  the  34th  percentile  of  the 
distribution. 

4.4. 2. 5.  Implementation. 

Figure  22  shows  examples  of  both  of  the  methods  explained  in  the  previous  sections.  In 
these  examples,  the  desired  percentile  (34th)  is  entered  as  a  constant  for  the  appropriate 
argument.  If  the  user  wants  the  flexibility  to  change  this  value,  it  should  be  stored  elsewhere  and 
referred  to  in  the  function  used  to  calculate  the  percentile. 
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Figure  22:  Examples  of  percentile  calculation. 


Example  A  in  Figure  22  shows  the  use  of  the  PERCENTILE  function  to  calculate  the 
34th  percentile  from  the  1991-2004  production  data.  Example  B  shows  the  use  of  the 
NORMINV  function  to  calculate  the  34th  percentile  from  the  sample  mean  and  sample  standard 
deviation.  Once  the  34th  percentile  is  determined,  it  can  be  incorporated  in  the  decision  for 
commissioning  mission  value  by  using  the  MAX  function.  The  most  concise  way  to  do  this 
would  be  to  replace  the  function  in  the  ‘Mission’  column  of  the  MDPM  (see  Figure  15)  with  the 
MAX  function,  with  the  existing  function  entered  as  one  of  the  arguments.  An  example  of  how 
to  do  this  is  shown  in  Figure  23. 
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Figure  23:  Using  the  Alternate  Minimum  Mission  Threshold. 

Here,  the  first  argument  in  the  MAX  function  is  the  absolute  minimum  of  6,  entered  as  a 
constant.  Our  34th  percentile  value  is  the  second  argument.  Note  that  we  have  used  the 
VLOOKUP  function  here  to  find  the  percentile  value  that  we  calculated  in  the  HIST  module. 

This  was  done  so  that,  if  the  data  in  the  HIST  module  is  sorted  differently  than  the  order  of 
schools  in  the  MDPM,  we  will  still  be  assured  of  getting  the  right  percentile.10  The  third 
argument  is  the  model  output  -  the  same  function  that  was  used  before  (see  Figure  15).  In  this 
example,  we  have  used  the  ROUND  function  to  round  the  results  of  the  functions  used  in  the 
second  and  third  arguments  to  the  nearest  integer.  This  compound  function  produces  the  desired 
result  -  it  sets  the  mission  value  at  either  6,  the  34th  percentile  of  the  particular  school,  or  the 
model  output  of  the  particular  school,  whichever  is  the  greatest. 

4.4.3.  Comparing  Mission  Numbers  to  the  Estimated  Production 
Distributions. 

Since  normal  distributions  were  used  to  estimate  the  production  distribution  for  each 
school,  those  distributions  can  be  used  to  provide  a  measure  of  an  assigned  commission  mission 
value  (either  from  the  model  or  from  negotiation)  with  respect  to  the  school’s  production  history. 

10  The  table  that  VLOOKUP  refers  to  in  this  example  has  ‘School  Name’  as  its  first  column  (column  1),  making  the 
column  containing  the  34th  percentile  (from  data)  column  19. 
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Also,  since  the  parameters  for  each  school’s  distribution  have  been  estimated  from  the  data,  the 
NORMDIST  function  can  be  used  to  determine  where  a  mission  value  ‘fits’  within  a  school’s 
production  history.  The  syntax  for  this  function  is  NORMDIST(x,  mean,  standard_dev, 
cumulative),  where  x  is  the  mission  value  we  want  to  evaluate,  mean  is  the  sample  mean  from 
the  production  data,  standard_dev  is  the  standard  deviation  from  the  production  data,  and 
cumulative  is  a  Boolean  variable  (when  set  to  TRUE,  the  function  returns  the  cumulative 
probability  P(X<x)).  This  function  can  quickly  be  applied  to  each  school,  and  the  results  used  to 
determine  if  any  mission  values  (either  from  model  output  or  post-negotiation)  appear  extremely 
unlikely  when  compared  to  a  school’s  production  history  (indicated  by  cumulative  probabilities 
either  near  zero  or  one).  These  schools  can  be  checked  again  to  ensure  that  they  are  receiving 
realistic  mission  values.11 


11  It  should  be  noted  that  this  is  intended  for  evaluation  only.  For  example,  a  school’s  mission  value  may  be  very 
high  compared  to  its  production  history,  resulting  in  a  cumulative  probability  near  one,  but  there  may  be  a 
compelling  reason  for  the  high  mission  value,  such  as  a  much  larger  recruitable  market  than  was  previously  known, 
for  which  that  school’s  ROTC  program  will  be  receiving  additional  resources  to  facilitate  greater  market 
penetration.  Hence,  the  primary  reason  for  this  tool  is  simply  to  identify  schools  that  receive  mission  values 
considered  extreme  when  compared  to  their  production  history. 
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Chapter  5:  Performance  Measures. 

5.1.  Measuring  Efficiency  with  Data  Envelopment  Analysis  (DEA). 

The  complexity  of  the  commissioning  process  and  the  number  of  factors  that  contribute 
to  the  successful  commissioning  of  a  certain  cadet  makes  the  use  of  simple  ratios  to  represent 
efficiency  of  very  limited  value.  There  are  several  factors  that  could  potentially  affect  the 
number  of  commissioned  officers  produced  by  a  certain  school.  These  include  the  assigned 
commissioning  mission,  the  number  of  cadre,  operating  budgets,  scholarships,  market 
characteristics,  space  constraints,  advertising,  and  so  on.  In  simpler  problems,  a  ratio  of  output 
to  input  can  be  used  as  a  measure  of  efficiency.  In  our  problem,  we  need  to  be  able  to  account 
for  all  the  relevant  inputs  and  outputs  when  attempting  to  measure  efficiency.  Data  Envelopment 
Analysis  (DEA)  makes  this  possible.  Unlike  regression  analysis,  which  compares  each  unit  of 
interest  against  the  central  tendency  of  the  whole,  DEA  finds  the  best  performers  based  on  the 
inputs  and  outputs  provided  and  compares  each  one  against  the  best  performers  (strictly 
speaking,  against  a  reference  set  that  is  identified  for  each  decision-making  unit,  or  DMU). 
Furthermore,  DEA  does  not  require  the  assignment  of  fixed  weights  to  the  various  inputs  (the 
values  of  which  are  typically  hard  to  justify)  -  weights  that  are  then  identically  applied  to  each 
DMU.  Instead,  the  weights  are  assigned  from  the  data,  and  each  DMU  is  assigned  a  “best  set”  of 
weights.  This  method  is  a  much  more  comprehensive  approach  to  separating  efficient  and 
inefficient  performers,  and  is  generally  perceived  as  a  fairer  approach  by  the  units  being 
examined. 

5.2.  The  Purpose  of  This  Chapter. 

It  is  not  our  intent  here  to  proceed  with  a  detailed  explanation  of  DEA.  For  a  more 
detailed  explanation  of  DEA,  see  (Cooper,  2000).  What  we  will  primarily  do  here  is  work 
through  an  example  dataset  using  a  DEA  software  application  called  Frontier  Analyst 
Professional,  in  order  to  demonstrate  how  it  can  be  used  to  provide  useful  information  regarding 
the  efficiency  of  ROTC  programs  in  the  near  term.  An  example  of  a  model  that  incorporates 
DEA  with  production  functions  can  be  found  in  (Brence,  2004). 

5.3.  An  Example  Using  Frontier  Analyst  Professional. 

To  demonstrate  the  process  used  in  DEA,  we  will  perform  an  analysis  on  an  example 
dataset  using  Frontier  Analyst  Professional  from  Banxia  Software.  The  dataset  used  in  this 
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example  was  derived  from  three  columns  of  data  stored  in  the  “CC  STC  Working  Base  Model” 
spreadsheet:  Under  “Historic  Production  Data”,  the  data  in  the  “3-Yr  Avg  Cmsn  Msn”  and  “3- 
Yr  Avg  Prod”  columns,  and  under  “Historic  Cadre”,  the  data  in  the  “Avg  #  of  Cadre”  column. 
This  analysis  consists  of  three  basic  steps  (adapted  from  the  Frontier  help  documentation): 

1)  Defining  and  selecting  the  units  (DMUs)  to  use  in  the  analysis. 

2)  Deciding  which  factors  to  use  for  inputs  and  outputs. 

3)  Running  the  analysis  and  interpreting  the  results. 

In  our  example,  the  schools  represent  our  DMUs,  and  we  will  filter  the  dataset  to  exclude  all 
HBCUs,  SMCs,  and  MJCs.  The  inputs  we  will  use  for  the  initial  analysis  are  “3-Yr  Avg  Cmsn 
Msn”  and  “Avg  #  of  Cadre”,  and  the  single  output  will  be  “3-Yr  Avg  Prod”. 

5.3.1.  Getting  Data  into  Frontier  Analyst. 

There  are  several  ways  to  get  your  data  into  Frontier  Analyst  (as  can  be  seen  in  the  dialog 
shown  in  Figure  24),  but  the  method  we  will  use  is  to  simply  paste  it  in  from  the  clipboard 
(which  is  done  by  selecting  the  first  option  as  shown): 


Figure  24:  Data  Input  Dialog 

For  this  example,  we  filtered  the  data  in  the  spreadsheet  (using  AutoFilter)  to  remove  the 
HBCUs,  MJCs,  and  SMCs,  then  copied  the  data  to  the  clipboard.12  This  removed  from  our 
example  dataset  the  primary  known  outliers  (there  will  be  a  small  group  of  other  outliers  that  we 
will  remove  individually  from  the  dataset  in  a  later  step).  Once  that  is  done,  we  clicked  ‘Next’ 
here  and  in  all  subsequent  dialogs  (since  no  other  changes  were  needed)  until  we  reached  the  end 
of  the  dialog.  Once  finished,  Frontier  Analyst  opens  the  Data  Viewer,  which  contains  our 

12  Actually,  there  is  an  additional  step  here.  Since  the  data  columns  in  Excel  are  not  contiguous,  they  were  copied 
and  pasted  into  a  blank  worksheet,  then  copied  from  there  and  pasted  into  Frontier.  Otherwise  we  got  an  error 
stating  we  were  pasting  too  many  variables. 
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dataset.  From  here  we  will  be  able  to  examine  the  dataset  and  make  decisions  as  to  which 
variables  need  to  be  identified  as  inputs  (controlled  or  uncontrolled)  and  outputs. 


Figure  25:  STC  data  in  the  Data  Viewer. 

The  data  that  we  pasted  into  Frontier  Analyst  included  the  school  names  along  with  the  three 
variables  of  interest.  The  data  can  also  be  edited  in  this  window  directly,  if  needed. 

5.3.2.  Setting  Inputs  and  Outputs. 

By  default,  Frontier  Analyst  identifies  all  the  variables  entered  into  the  data  viewer  as 
controlled  inputs13.  Variables  can  also  be  identified  as  uncontrolled  inputs  or  as  outputs.  An 
uncontrolled  input  would  be  an  input  that  we  have  no  power  to  increase  or  decrease  (an  example 
from  business  might  be  the  number  of  competitors  in  a  given  area).  For  this  example  we  assume 
that  all  input  variables  are  controlled  inputs,  so  we  will  not  change  them.  However,  we  need  to 
change  the  variable  “3-Yr  Avg  Prod”  to  an  output.  To  do  this,  you  simply  click  in  any  one  of  the 
cells  in  that  column  and  then  change  the  Input-Output  type  for  that  variable  by  selecting  “output” 
from  the  pull-down  menu  (see  Figure  26). 


13  Controlled  and  uncontrolled  inputs  are  referred  to  discretionary  and  non-discretionary  inputs  in  (Cooper, 
2000:63). 
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Input/Output  name:  3-yr  Avg  Prod  Input/Output  type: 

Controlled  input  T 

HI  M 

'e 

3-yr  Avg  Cmsn  Msn 

3-yr  Avg  Prod 

Avg  #  of  Cadre 

15.00 

7.00 

23.00 

19.67 

9.00 

13.33 

11.00 

6.00 

12.00 

11.67 

7.00 

17.00 

10.00 

7.00 

[Uncontrolled  input 
[Output 


Input/Output  name:  3-yr  Avg  Prod  Input/Output  type: 


Active 

3-yr  Avg  Cmsn  Msn| 

3-yr  Avg  Prod 

Avg  #  of  Cadre 

f  F 

15.00 

13.00 

7.00 

F 

23.00 

19.67 

9.00 

F 

13.33 

11.00 

6.00 

F 

12.00 

11.67 

7.00 

F 

17.00 

10.00 

7.00 

J2 

_ 

LLiLL 

x  nn 

Figure  26:  Identifying  the  output  variable. 


The  column  background  color  changes  to  identify  it  as  an  output  variable.  There  were 
also  three  specific  schools  that  we  decided  to  remove  from  the  dataset  (they  were  removed 
because  they  contained  limited  historical  data  in  the  1991-2004  data).  This  was  done  directly  in 
the  Data  Viewer  by  selecting  the  schools  and  then  clicking  the  “Deactivate  Current  Unit”  button: 


Deactivate  Current  Unit 


o  s 


Uni  name:  UNIV  OF  MARYI  ^  Input/Output  name:  Avg  #  of  Cadre  Input/( 


Unit  Name 

Active 

3-yr  Avg  Cmsn  Msn 

3-yr  Avg  Prod 

Avg  #  of  Cadre 

UNIV  OF  ILLINOIS  AT  CHICAGO 

F 

25.00 

13.67 

10.00 

► 

UNIV  OF  MARYLAND  COLLEGE  PAP 

F 

0.00 

0.00 

UNIV  OF  MINNESOTA-TWIN  CITIES 

F 

17.00 

13.67 

6.00 

UNIV  OF  MISSISSIPPI 

F 

11.00 

8.33 

6.00 

Figure  27:  Removing  an  individual  school  from  the  dataset. 


This  action  causes  these  DMUs  to  be  removed  from  the  analysis  (their  removal  is 
indicated  by  graying  out  the  text  -  they  are  not  deleted  from  the  data  viewer).  Once  all 
modifications  to  the  data  have  been  done,  we  need  to  decide  what  type  of  DEA  analysis  we  want 
to  perform  with  respect  to  optimization  type  and  scaling  type.  These  selections  are  made  through 
the  DEA  options  dialog  (shown  in  Figure  28). 
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Optimisation  mode 


Min  In 


Seekto  minimise 
inputs  to  produce  the 
same  outputs. 


^LJ 

Max  Out 


Seekto  maximise 
outputs  given  the 
current  inputs. 


Scaling  moc 

Je 

Outputs  directly  reflect  ,  Outputs  fall  off  as  input 

input  levels,  (i.e.  v  levels  rise.  (i.e. 

1  doubling  input  Varying  doubling  input 

produces  exactly  '  produces  less  than 

CCR  mode-""'1  BCC  mode'""'1 

L/_ 

Constanl 

Substitute  Zero  values  with:  |0.001 

lAdvanced  >>| 

Figure  28:  The  DEA  Options  dialog. 


Two  different  types  of  DEA  models  are  reflected  in  the  scaling  mode  selections  in  the 
above  dialog.  The  more  basic  of  the  two  is  the  CCR  model,  initially  proposed  by  Chames, 
Cooper,  and  Rhodes  in  1978.  A  detailed  explanation  of  the  CCR  model  can  be  found  in  chapters 
2  and  3  of  (Cooper,  2000).  The  other  alternative  is  the  BCC  model  (Banker-Chames-Cooper) 
model  which  is  characterized  by  variable  returns  to  scale  (see  Chapter  4  of  (Cooper,  2000)  for  a 
detailed  explanation  of  this  model).  The  other  consideration  is  in  the  optimization  mode  we 
choose.  For  this  example,  we  will  first  seek  to  maximize  outputs14  and  assume  constant  returns 
to  scale.  In  another  example,  we  will  use  the  BCC  model  which  assumes  variable  returns  to 
scale. 

5.3.3.  Analysis  and  Results  Using  the  CCR  Model. 

When  we  run  the  analysis,  the  first  results  we  see  are  the  efficiency  scores.  Schools  that 
receive  a  score  of  100  are  considered  efficient.  Those  that  receive  a  score  of  less  than  100  are 
considered  inefficient  (the  score  is  the  percent  of  efficiency  relative  to  one  or  more  of  the 
efficient  school  that  make  up  the  reference  set  for  the  inefficient  school). 


14  Referred  to  as  an  output-oriented  model  in  (Cooper,  2000:58,  103). 
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Figure  29:  The  Efficiency  Scores  table. 


In  this  example,  only  Central  Missouri  State  University  and  the  University  of  Hawaii  at  Manoa 
are  considered  efficient.  A  graphical  summary  of  efficiency  scores  is  available,  showing  a 
histogram  of  schools  that  scored  within  a  certain  range. 


Figure  30:  The  Scores  Distribution  chart. 


We  can  see  here  that  the  bulk  of  the  schools  (153  out  of  235)  in  our  dataset  have  efficiency 
scores  between  41  and  70.  Another  way  to  see  this  distribution  is  by  examining  the  frontier  plot, 
which  plots  each  school  relative  to  the  efficient  frontier. 
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Figure  31:  The  frontier  plot. 

The  efficient  frontier  always  touches  at  least  one  point.  In  this  case  the  two  points  on  the 
frontier  are  the  two  school  identified  in  the  efficiency  scores  report  at  efficient.  The  region 
above  and  to  the  right  of  the  efficient  frontier  (where  all  of  our  other  data  points  reside) 
represents  the  production  possibility  set.  Each  point  not  on  the  frontier  has  its  efficiency 
measures  relative  to  a  reference  set  that  consists  of  at  least  on  of  the  points  on  the  efficient 
frontier  (possibly  both).  In  general,  the  further  a  school  is  from  the  efficient  frontier,  the  lower 
its  efficiency  score.  To  see  what  other  useful  information  can  be  gleaned  from  this  analysis,  we 
will  examine  a  couple  of  school  from  the  dataset  more  closely. 

5.3.3. 1.  Example  School  1:  Appalachian  State  University. 

Appalachian  State  University  received  an  efficiency  score  of  56.9  percent.  This  places  it 
firmly  in  the  center  of  the  distribution  of  efficiency  scores.  Its  reference  set  is  Central  Missouri 
State  University. 
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Frontier  Analyst  also  gives  us  additional  tools,  one  of  which  allows  us  to  examine 
potential  areas  of  improvement.  This  information  is  provided  by  default  in  graphical  form,  but 
the  numerical  values  can  also  be  shown  in  a  table  (the  entire  graph  window  and  part  of  the  table 
window  are  shown  in  Figure  33  -  the  two  different  representations  can  be  selected  by  using  the 
buttons  on  the  left  side  of  the  window). 


Input/ Output _ Actual _ T  arget  Poterrti  al  I  m  prove  m  e  nt 

Avg#  of  Cadre  7 _  7  0 

3-yr  Avg  Cfinsn  Msn  15  14  -6,67 

3-yr  Avg  Rrod_ 13_ 22M_ 75  86 

Figure  33:  Potential  improvements  report  (graph  and  table)  for  Appalachian  State. 

Note  that  this  is  not  the  only  possible  way  to  improve  the  efficiency  of  Appalachian  State. 
In  fact,  if  the  suggested  improvements  were  made  exactly  as  indicated  (by  achieving  the  targets 
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for  each  variable),  it  would  place  Appalachian  State  on  the  efficient  frontier  precisely  where 
Central  Missouri  State  is  located  (since  Central  Missouri  State  is  the  only  school  in  its  reference 
set).  In  truth,  any  adjustment  that  places  Appalachian  State  University  on  the  efficient  frontier 
will  make  the  school  efficient  and  place  it  among  the  reference  set.  Specifically,  it  would  need 
to  meet  one  of  the  following: 


1)  Avg#of Cadre  m  for  3- YrAvgCmsnMsn  ^Q  ^12 


3-  Yr  AvgProd 


3- Yr  AvgProd 


3  -  Yr  AvgCmsnMsn  =  Q  5??  foj.  Avg#  of  Cadre  >  Q  ^ 
3 -Yr  AvgProd  3 -Yr  AvgProd 


3)  On  the  line  3- Yr  AvgCmsnMsn  = 
3- Yr  AvgProd 

for  0.306  <  Ayg#ofCadre  <0.346 
3- Yr  AvgProd 


f  Avg#  of  Cadre  A 


3- Yr  AvgProd 


+  0.623 


Equation  3:  Mathematical  description  of  the  efficient  frontier. 


Of  course,  any  adjustment  that  causes  Appalachian  State  University  to  outside  of  the 
production  possibility  set  would  cause  it  to  replace  one  or  both  of  the  schools  in  the  current 
reference  set,  assuming  there  are  no  changes  elsewhere.  Frontier  Analyst  also  provides  a 
reference  comparison: 


Figure  34:  By-variable  comparison  to  a  reference  set  DMU. 
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This  graph  shows  a  direct  comparison  for  each  variable  between  Appalachian  State 
University  and  Central  Missouri  State  University.  The  numbered  labels  to  the  right  of  each  bar 
for  the  reference  indicate  a  percentage  comparison  to  the  school  being  examined.  In  this  case,  it 
shows  that  Central  Missouri  State  has  71%  of  the  average  cadre  and  66%  of  the  average  mission 
that  Appalachian  State  did  for  the  same  period,  but  produced  125%  of  Appalachian  State’s 
average  production. 

5. 3. 3. 2.  Example  School  2:  University  of  Portland. 

The  University  of  Portland  received  an  efficiency  score  of  8 1 .4%,  placing  it  in  the  upper 
tail  of  the  distribution  of  efficiency  scores.  It  has  both  efficient  schools  in  its  reference  set. 


Figure  35:  Comparison  of  the  University  of  Portland  to  its  reference  set  on  the  frontier  plot. 

The  only  potential  improvement  that  Frontier  Analyst  recommends  is  to  increase  production  by 
22%.  This  adjustment  would  place  the  point  in  the  frontier  plot  at  the  other  end  of  the  blue  line 
segment  shown  in  the  above  plot. 
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- 

Increasing  production  by  22% 
moves  the  data  point  to  here.' 


Figure  36:  The  effect  of  implementing  the  potential  improvements. 


When  we  look  at  the  reference  comparison,  there  are  now  two  graphs  available,  since 
there  are  two  schools  in  the  reference  set.  Each  comparison  shows  the  relative  percentages  of  the 
respective  reference  school  when  compared  with  the  school  of  interest.  In  both  graphs,  the 
University  of  Portland  is  represented  in  blue. 


Figure  37:  By-variable  comparison  of  the  University  of  Portland  to  Central  Missouri  State  University. 
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Figure  38:  By-variable  comparison  of  the  University  of  Portland  to  the  University  of  Hawaii  at  Manoa. 

These  graphs  demonstrate  a  scale  difference  between  the  two  reference  schools,  which  is 
revealed  when  compared  with  the  University  of  Portland.  The  University  of  Hawaii  at  Manoa, 
compared  in  the  graph  at  bottom,  has  larger  values  for  every  variable  compared  to  the  University 
of  Portland,  while  Central  Missouri  State  has  smaller  values  for  every  variable.  In  both  cases  the 
schools  in  the  reference  set  show  their  highest  comparison  percentages  in  the  area  of  production. 
5.3.4.  Analysis  and  Results  Using  the  BCC  Model. 

Under  the  CCR  model,  constant  returns  to  scale  is  assumed.  This  essentially  means  that  a 
doubling  of  all  inputs  leads  to  a  doubling  of  all  outputs.  If  the  DMUs  do  not  exhibit  this 
behavior,  then  they  are  exhibiting  variable  returns  to  scale.  Because  of  this,  when  we  use  the 
BCC  model,  DMUs  are  compared  in  terms  of  efficiency  to  other  DMUs  of  the  same  or  similar 
scale  as  opposed  to  every  DMU  in  the  data  set.  DMUs  will  be  reported  to  be  at  least  as  efficient 
as  they  were  under  the  CCR  model,  and  some  DMUs  will  be  reported  as  more  efficient.  To  run 
this  analysis,  the  only  item  that  needs  to  be  changed  is  the  scaling  mode  in  the  analysis  options: 
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Figure  39:  Changing  to  varying  returns  to  scale  (BCC  model)  in  the  DEA  options  dialog. 


We  now  select  varying  returns  to  scale,  which  will  then  utilize  the  BCC  model.  We  will 
maximize  outputs,  as  before.  After  running  the  analysis,  we  again  see  the  efficiency  scores  as 
the  first  output: 


Ol  Efficiency  Scores 


Details  Edit  data 
235  units. 


Illlllii.  Hlllhi.  Illlllih 

Show  all  100%  <100% 


^  Azl 

Unsort  SortA-Z 


»ll  1 9  i 

Sort  9-1  Sort  1-9 


□ 

Summary 


IlMame 

Score 

Scale 

EDINBORO  UNIV  OF  PENNSYLVANIA 

100.00 

constant  ? 

GEORGE  MASON  UNIVERSITY 

100.00 

constant 

SOUTHERN  ILL  UNIV-EDWARDSVIL 

100.00 

constant 

AUGUSTA  STATE  UNIVERSITY 

100.00 

constant 

CENTRAL  MISSOURI  STATE  UNIV 

100.00 

constant 

UNIV  OF  HAWAII  AT  MANOA 

100.00 

constant 

UNIVERSITY  OF  PR  MAYAGUEZ 

100.00 

constant 

UNIVERSITY  OF  GUAM 

99.23 

decreasing 

UNIV  OF  CALIFORNIA-DAVIS 

96.60 

increasing 

THE  JOHNS  HOPKINS  UNIVERSITY 

95.42 

increasing 

CLARKSON  UNIVERSITY 

93.67 

increasing 

THE  PENNSYLVANIA  STATE  UNIV 

91.97 

increasing 

UNIVERSITY  OF  NOTRE  DAME 

91.92 

increasing 

TEXAS  STATE  UNIV-SAN  MARCOS 

91.87 

increasing 

UNIVERSITY  OF  CENTRAL  FLORIDA 

90.12 

increasing 

BRIGHAM  YOUNG  UNIVERSITY 

89.44 

increasing 

CAMPBELL  UNIV 

88.52 

increasing 

XAVIER  UNIVERSITY 

87.25 

increasing 

NIAGARA  UNIVERSITY 

85.42 

decreasing 

AUSTIN  PEAY  STATE  UNIVERSITY 

84.82 

increasing 

WASHINGTON  UNIVERSITY 

84.56 

decreasing 

UNIVERSITY  OF  PORTLAND 

84.53 

increasing 

Figure  40:  Efficiency  Scores  table  from  the  BCC  model. 


When  the  BCC  model  is  used,  an  additional  column  appears  labeled  “Scale”.  This 
indicates  the  type  of  returns  to  scale  that  a  particular  school  exhibits.  The  University  of  Guam 
shows  a  decreasing  return  to  scale,  which  mean  than  a  doubling  of  all  inputs  leads  to  something 
less  than  a  doubling  of  outputs.  Likewise,  the  University  of  California  at  Davis  shows  an 
increasing  return  to  scale,  which  means  that  a  doubling  of  all  inputs  leads  to  something  more 
than  a  doubling  of  outputs.  Note  that  there  are  several  more  schools  that  are  identified  as 
‘relatively  efficient’  than  we  had  under  the  CCR  model.  This  is  a  typical  result  of  changing  to  a 
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varying  retums-to-scale  model  from  a  constant  retums-to-scale  model.  Overall,  however,  the 
distribution  of  efficiency  scores  has  not  changed  much  from  the  CCR  model  (compare  Figure  41 
to  Figure  30). 


Figure  41:  Distribution  of  Scores  chart  from  the  BCC  model. 

5.3.5.  A  Broader  View  at  Future  Application. 

In  actual  use,  there  are  more  inputs  that  have  an  effect  on  production  than  the  two 
included  in  our  example.  Several  of  these  were  listed  in  the  introduction  to  this  chapter.  In 
general,  it  is  possible  to  “inflate”  the  number  of  DMUs  that  will  be  identified  as  efficient,  but 
normally  this  becomes  a  problem  when  the  number  of  DMUs  is  less  than  the  total  of  the  input 
and  output  variables  (Cooper,  2000: 103).  This  would  not  be  a  problem  if  we  are  examining  all 
of  the  schools  at  once,  but  should  be  considered  if  the  data  is  examined  in  subsets.  One  way  to 
reduce  the  number  of  variables  in  the  analysis  is  to  remove  inputs  that  are  strongly  correlated 
with  another  input.  Frontier  Analyst  has  a  tool  to  assist  with  this  -  once  the  analysis  has  been 
run,  you  can  create  a  scatterplot  (Frontier  calls  it  an  x-y  plot)  of  any  two  variables  in  the  data  set. 
The  window  includes  a  correlation  coefficient  for  the  two  variables  being  examined  (shown  in 
the  upper-right  comer  of  Figure  42). 
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Figure  42:  An  x-y  plot  of  two  input  variables. 

A  detailed  analysis  of  USAREC  recruiting  practices,  along  with  the  development  of  a 
production  function  that  incorporates  DEA  is  given  in  (Brence,  2004).  The  regression  model 
explained  in  section  3.2.1  of  that  paper  is  roughly  analogous  to  the  current  STC  model,  although 
the  variables  used  to  determine  the  recruitable  market  are  necessarily  different  (as  are  the  time 
frames  of  interest).  The  addition  of  DEA  into  the  model  was  done  by  analyzing  the  data  of 
several  inputs  relevant  to  recruiting  and  recruiter  allocation.  If  EFSACC  wishes  to  examine 
alternate  models  (with  possibly  greater  fidelity)  once  the  OCMPS  is  complete,  an  approach 
similar  to  the  one  taken  in  (Brence,  2004)  could  be  explored. 
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Chapter  6:  Conclusion 

The  STC  model  uses  an  approach  that  makes  sense  with  respect  to  USACC’s  current 
business  practices.  Its  philosophy  is  consistent  with  the  need  to  assign  mission  numbers  to 
ROTC  detachments  in  accordance  with  recruitable  markets  assessments.  However,  there  is  still 
enough  variability  in  the  recruitable  market  estimates  to  keep  the  model  in  its  current  role  (a 
starting  point  that  can  be  negotiated),  as  opposed  to  one  with  enough  fidelity  to  eliminate  the 
need  for  extensive  mission  adjustments.  It  is  possible  that,  as  the  OCMPS  proceeds,  this 
variability  will  be  reduced,  or  new  and  better  indicators  of  recruitable  market  size  may  present 
themselves.  Until  that  time,  we  have  geared  our  recommendations  toward  three  short-term  goals. 
First,  simplify  the  implementation  of  the  model  to  make  it  leaner  and  easier  to  work  with. 

Second,  utilize  built-in  Excel  functions  wherever  possible  to  reduce  or  eliminate  mathematical 
errors,  incorrect  value  assignments,  and  data  duplication.  Finally,  utilize  historical  production 
data  to  assign  different  minimum  mission  values,  where  appropriate,  in  order  to  assist  in  meeting 
the  increased  overall  mission.  Beyond  the  near  term,  we  also  recommend  applying  the  DEA 
approach  to  available  data  in  order  to  separate  efficient  and  inefficient  performers.  This  will 
provide  additional  data  on  which  to  base  resource  allocation  decisions. 
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Appendix  A:  An  Explanation  of  the  Ryan- Joiner  Test  for 

Normality 


A  normal  probability  plot  is  often  used  to  determine  whether  it  is  plausible  that  a  certain 
set  of  data  was  sampled  from  a  normal  population.  This  is  done  by  ordering  the  sample  x's 

from  smallest  to  largest,  (x  x  . .  and  then  plotting  the  points  [x{l),yl ),  where 

y,  =  O  '((/  -  0.5 )/«).  The  closer  this  plot  resembles  a  straight  line,  the  more  plausible  it  is  that 

the  sample  was  taken  from  a  normal  distribution.  This  method  is  easy  to  use,  but  can  become 
cumbersome  if  you  have  to  repeat  it  more  than  a  few  times.  The  Ryan-Joiner  test  for  normality 
allows  us  to  take  a  quantitative  measure  of  the  extent  to  which  points  cluster  about  a  straight  line. 
A  hypothesis  test  is  used  to  make  a  determination  based  on  a  chosen  significance  level.  The 
details  are  explained  here: 


Let  y.  =  O  1  [(i  -  0.375)/(n  +  0.25)]  and  compute  the  sample  correlation 
coefficient  r  for  the  n  pairs  (x(1) ,  yx ),...,  (x,n),  yn )  The  Ryan  -  Joiner  test  of 
Hr  :  the  population  distribution  is  normal 
versus 

H A  :  the  population  distribution  is  not  normal 
consists  of  rejecting  H0  when  r<ca  for  significance  level  a. 


A  more  detailed  explanation  can  be  found  in  Devore,  page  65 1 . 
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Appendix  B:  An  Detailed  Explanation  of  the  Simulation 
Used  to  Determine  the  Alternate  Minimum  Mission 

Threshold 


Doing  Monte  Carlo  simulation  in  Microsoft  Excel,  where  random  values  are  simulated 
from  a  normal  distribution,  is  a  straightforward  affair.  You  can  generate  these  random  values 
with  a  compound  function  that  utilizes  the  NORMINV  and  RAND  functions  with  the  following 
statement: 

=NORMINV(RAND(),0,1 ) 


In  this  case,  we  have  simulated  a  value  from  the  standard  normal  distribution  (see 
Microsoft  Excel  Help  for  details  on  the  syntax  of  these  functions).  In  order  to  simulate  500 
values  from  this  distribution,  this  function  can  be  copied  into  500  cells  -  the  RAND  function  will 
produce  a  different  pseudorandom  probability  each  time  it  is  called,  which  then  gives  a  new 
simulated  value  from  the  distribution.  For  our  purposes,  we  needed  the  simulation  to  only  draw 
random  sample  values  from  a  portion  of  the  normal  distribution  used  to  approximate  each 
school’s  production  -  specifically,  only  values  that  were  greater  than  or  equal  to  a  specified 
percentile.  To  do  this,  we  used  the  RANDBETWEEN  function  (instead  of  the  RAND  function) 
to  generate  the  desired  random  probabilities. 
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20.21 

5.34 

0.773 

25 
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In  our  simulation,  the  value  stored  in  cell  G275  is  an  integer  between  1  and  999,  and  is 
controlled  by  the  slider  at  the  top  of  column  G.  The  function  is  the  same  in  each  row  and 
generates  random  probabilities  between  the  threshold  (G275/1000)  and  0.999.  This  value  is 
then  used  in  determining  the  production  values  for  the  simulation  -  this  is  done  with  another 
compound  expression  that  uses  the  MAX,  ROUNDUP,  and  NORMINV  functions: 


B  £  SliBlilt  7.  ip 


T  1 1  .j 


=IF(ISERROR(NORMINV(G2,E21F2)),"',MAX(6,ROUNDUP(NORMINV(G2,E2,F2),0))) 
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The  first  part  of  the  function  uses  the  IF  and  ISERROR  functions  to  deal  with  blank  cells 
or  missing  data.  The  function  that  actually  generates  our  random  production  values  is  outlined  in 
read.  The  effect  of  this  function  is  to  generate  production  values  that  are  always  greater  than  a 
school’s  34th  percentile,  and  are  never  less  than  six.  The  values  in  column  H  are  totaled  to 
produce  a  value  for  overall  production. 

In  order  to  determine  the  frequency  of  total  production  values  that  exceed  4400,  we  need 
to  run  this  simulation  several  times.  You  can  manually  do  this  by  pressing  the  F9  key  (which 
will  recalculate  the  entire  worksheet),  but  it  would  be  tedious  to  do  this  500  times  and  write  the 
results  down  after  each  run.  There  are  specialized  Excel  add-ins  that  can  handle  this  task  (such 
as  Palisade’s  @Risk  for  Excel® ),  but  it  can  be  done  with  a  “trick”  that  uses  Excel’s  table 
function.  The  steps  are  as  follows: 

1 .  Use  the  SUM  function  to  calculate  the  total  production  of  all  schools.  An  example  of  this 
is  shown  below,  using  data  from  10  schools  (the  function  is  stored  in  cell  El 2). 
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i  a j  ,j  a  y  .a  * 

-  10  *  B  I  U 


$ 


2  -r  fi,  =SUM(E3:E11) 

v |  B  |  C  |  d\  1  E 


2.  Make  a  reference  to  the  summation  cell  in  another  cell  on  the  worksheet  (i.e  enter  “=E12) 


in  another  cell. 


I  U  li:  =  =  : g  1  $  % J 

*  -  = 

=E12 

o 

m 
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G 

H  1 

Probability  Production 

=E12 

1 

0.928  20 

0.772  9 

0.741  17 

0.41  19 

0.72  13 

0.912  17 

0.482  18 

0.731  10 

0.611  16 

3.  Starting  one  cell  below  and  to  the  right  of  the  last  cell,  start  a  sequence  of  integers 
beginning  with  1  and  ending  with  the  highest  number  of  simulation  runs  you  want  to 
execute  (we’ll  use  10). 
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9 

TOTAL  141 

10 

w\ 

4.  Highlight  the  block  of  cells  as  illustrated  below. 
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Select  “Table..”  from  the  Data  menu.  The  Table  dialog  appears  as  shown  below. 
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TOTAL  141 


i 

Table  \4 

Row  input  cell:  |  |^«| 

Column  input  cell:  |  |^L*| 

OK  |  |  Cancel 

6.  Click  in  the  “Column  input  cell”,  then  click  in  an  empty  cell  (one  you  know  you  will  not 
use  later).  We’ll  use  the  cell  just  below  the  sequence  for  this  example. 
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Probability 

Production 
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0.652  23 
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TOTAL 
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I  * 

Row  input  cell: 
Column  input  cell: 


OK 


$G$13 

Cancel 


7.  Click  “OK”.  The  simulation  will  start  (if  you  have  AutoCalculate  turned  off,  you  will 
have  to  press  the  F9  key).  The  cells  to  the  right  of  the  sequence  will  populate  with  results 
from  individual  simulation  runs.  This  process  can  be  slow  -  in  this  example,  we  only  ran 
10  simulation  runs,  and  Excel  populated  the  cells  in  about  three  seconds  (on  a  2GHz 
Pentium  M).  In  the  actual  simulation  (which  we  ran  500  times),  run  time  was  about  90 


8.  In  our  simulation,  we  were  interested  in  the  proportion  of  simulation  runs  that  resulted  in 
a  total  production  of  at  least  4400.  This  can  be  done  using  the  COUNTIF  function.  In 
this  example,  we  will  look  for  simulation  runs  that  resulted  in  a  total  of  at  least  140.  You 
can  do  this  with  the  following  expression: 
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Function  is  in  this  cell  H  I 

1  8| 

IS 

To  get  the  proportion  of  runs  meeting  the  criteria,  just  divide  the  expression  by  the 
number  of  runs  (i.e.  t6=COUNTIF(H3:H12,”>=140”)/10). 
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Appendix  C:  Table  of  Abbreviations 


ADM 

Academic  discipline  mix 

BCC 

Banker-Chames-Cooper  DEA  model  (variable  returns  to  scale) 

CCR 

Chames-Cooper-Rhodes  DEA  model  (constant  returns  to  scale) 

DEA 

Data  Envelopment  Analysis 

DMU 

Decision-making  unit 

HBCU 

Historically  black  colleges  and  universities 

HIST 

Historical  Data  Module 

MDPM 

Missions  Data  Processing  Module 

MJC 

Military  Junior  College 

NCES 

National  Center  for  Education  Statistics 

OAS 

Officer  Accession  Strategy 

OCMPS 

On-Campus  Market  Potential  Study 

ODM 

OCPMS  Data  Module 

ORCEN 

Operations  Research  Center  of  Excellence 

ROTC 

Reserve  Officer  Training  Corps 

SAL 

Scholar- Athlete-Leader 

SDM 

School  Data  Module 

SMC 

Senior  Military  College 

STC 

Shaping  the  Cohort 

SUM 

Data  Summary  Module 

US  ACC 

United  States  Army  Cadet  Command 

USAREC 

United  States  Army  Recruiting  Command 

YATS 

Youth  Attitude  Tracking  Study 
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